Smoothed Bootstrap and Statistical Data Cloning for Classifier Evaluation

This paper is concerned with the estimation of a classifier’s accuracy. We present a number of novel bootstrap estimators, based on kernel smoothing, that consistently show superior performance on both synthetic and real data, with respect to other established methods. We call the process of (re)sampling the data via kernel-based smoothed bootstrap data cloning. The new cloning methods outperform cross-validation and the .632+ bootstrap, which, according to Efron and Tibshirani, is the estimator of choice. Finally, we extend our estimators to complex real-life data sets, in which a data point might include real, bounded, integer and nominal attributes, thus allowing for better classifier evaluation over limited real data repositories such as the UCI repository.